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(57) Abstract: A prosthesis imaging and migration tracking 
method and apparatus that analyses two-dimensional images, 
typically X-rays, to determine the position and orientation of 
a three-dimensional object such as prosthesis. A three-dimen- 
sional representation of the object of the implanted three-di- 
mensional object is matched to a silhouette, so as to estimate 
its pose. A scaling factor can also be determined to indicate 
reasonably accurate distances. Once the pose of the known 
three-dimensional object is determined, the surrounding bone 
can also be identified in the two-dimensional image using ob- 
ject determination techniques or matching with historical or 
model information of the other object. Knowledge of the three- 
dimensional pose of the objects is used for determining the 
relative complex movement of the objects by comparing their 
poses over time. Pictorial and graphical records can than be 
generated to assist clinical assessment of the bone and prosthe- 
sis union. The method and apparatus of the invention can also 
be used on historical two-dimensional images and the larger 
the number of images, even those having different aspects of 
the same limb, the better the assessment of the migration of 
the objects identified. 
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2-D AND 3-D POSE ESTIMATION OF ARTICLES FROM 2-D IMAGES 

This invention relates to the automatic analysis of two-dimensional images to 
determine the spatial orientation of a three-dimensional object within the two- 
dimensional image. In particular this invention can be applied to the analysis of the 
bonding of medically implanted orthopedic prosthesis in human bone. Furthermore, 
over time the degree of loosening and migration of the implant relative to the bone 
can be quantified. It is also possible using the apparatus and method of the invention 
to assist in the classification and assessment of bone fractures. 

BACKGROUND 

This invention is applicable to image analysis the image itself being of any 
subject matter. However, for illustrative purposes only, this specification will 
describe image analysis in the medical field and in particular, the analysis of 
radiographic images of the human skeleton involving an implanted prosthesis. The 
medical procedures developed by surgeons for orthopaedic prosthesis implantation 
are now well developed and many bones and joints of the body damaged by disease 
and injury can now be replaced or supplemented. 

The surgeon however, needs to maintain a constant watch over the 
development of the bonds that form between the implanted prosthesis and the 
natural bone. The practitioner is ever mindful of the possibility of the bonds not 
being formed correctly or sufficiently. The result of a less than acceptable bonding of 
the bone to the prosthesis or an avoidable misalignment, can range from a failure of 
the supplemented joint or bone to support the patient's weight and consequent 
collapse of the patient, to minor patient discomfort. 

Clearly, the importance of reliable analysis of the bonding between bone and 
prosthesis requires the use of a combination of many clinical tools. 
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The primary and cheapest tool is two-dimensional X-rays of the patient. These 
however can not be easily interpreted to determine with certainty the actual 
orientation of the implant with respect to the bone and more particularly their 
relative orientations from one X-ray to the next. Furthermore there can be a lack of 
consistency in the interpretation of these factors by different clinicians. More difficult 
again is the assessment of the relative movement of the prosthesis and bone over 
time. Nonetheless, bone growth about the prosthesis can make image comparison 
difficult if not totally subjective in some circumstances. 

The next, but much more expensive tool is three-dimensional tomography of 
the patient. CAT scans being the typical apparatus used for this task will readily 
identify the three-dimensional form of both the implant and the bone. Not 
surprisingly this tool is used less than would be desired because it is so expensive. 

Experienced clinicians skilled in the interpretation of X-rays and three- 
dimensional scans of the implant and associated bone can use these tools to improve 
the accuracy and usefulness of their assessment of the bone to implanted prosthesis 
bonding process and determine which of numerous levels of intervention are 
required. The earlier abnormalities can be assessed the more likely it is that remedial 
work can be done to rectify the developing problem. 

Some levels of non-clinician assisted analysis is conducted by scan operators 
manually identifying consistently recognizable points or regions (landmarks) on the 
implant and bone after which it is possible to measure relative movements if they 
have occurred. Unfortunately, this approach is liable to also repeat the same errors 
committed by clinicians. 

It is known that early intervention if properly assessed has benefits for 
patients as well as the community because more expensive and less debilitating 
medical procedures are avoided. 
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In the worst case, the implant can be the cause of further damage to the bone. 
If the bone is usable again there may be a need for a further operation/s to extract the 
implant. A new implant is inserted or pinned on to the bone as the case may be. 
Importantly, frequent and reliable assessment can greatly lessen the chance that this 
situation will arise. 

In the best case, the clinician may recommend more strenuous physiotherapy 
or advise the patient they can resume normal activity with the affected joint or limb. 

Of importance also is the assessment of bone fracture in good bone of the 
affected limb or in the bone surrounding an implant caused in part by the lack of or 
incorrect bonding between bone and prosthesis. 

Fractures are more difficult to identify. Hence, the variability of the assessed 
degree of fracture by the same clinician can be inconsistent. While there is also a 
likelihood that the same patient when assessed by different clinicians will have 
different fracture assessments even when the same tools are used. Unfortunately, 
human error and interpretation will always be present when two-dimensional and 
even three-dimensional imaging are the only tools available. 

The migration of the prosthesis overtime longitudinally, transversally and 
rotationally is particularly difficult to assess when only a two-dimensional X-ray 
view is available. Such X-ray views are sometimes referred to as an Anterior 
Posterior (AP) radiograph and even with the use of filters and software, accurate 
migration analysis due to distortions caused by patient positioning can still occur. 

An Ein Bild Roentgen Analyse (EBRA) method has been developed to reject 
non-comparable AP radiographs and provide some independence from the skill of 
the clinician to provide repeatable and reliable analysis. A further arrangement called 
Roentgen Stereophotgr Ammetry (RSA) involves the use of two radiographs to create 
a stereographic representation of the implant and limb one or both of which have X- 
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ray opaque landmarks fitted or implanted during surgery. The analysis apparatus 
and the surgical elements are very expensive to purchase, to maintain and still 
requires accurate patient orientation during the recording process. The apparatus 
does not allow for the analysis of historical X-ray images. 

It is an aim of the invention to provide a tool that overcomes or lessens some 
at least of the problems described above. 

Preferably the method and apparatus of the invention will in part at least 
automate and make more consistent the assessment of two-dimensional images of 
implants in bone structures of bonding and fracture characteristics. 

It is also preferable that assessment information is provided to the clinician in 
a format that shows trends and highlights features for further consideration. The 
system should be able to measure distances so that threshold distances, determined 
from an expert system environment, if exceeded can identify to clinicians, areas of 
the image deserving of further investigation and provide a way to display and thus 
analyze trends of migration over time. 

BRIEF DESCRIPTION OF THE INVENTION 

In a broad aspect of the invention A method for analysis of a two-dimensional image 
to determine the position of a three-dimensional object therein consisting of the steps; 
a) determining the form of said three-dimensional object in the two-dimensional 
image; b) determining the pose of a three-dimensional representation of said object 
that conforms to the form of the said three-dimensional object in the two-dimensional 
image; c) determining a first landmark with reference to said three-dimensional 
object; d) determining a second landmark on an other object in said two-dimensional 
image; and e) determining the distance between said landmarks. 
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In a yet further aspect of the invention a method of analysis of a two- 
dimensional image to determine the position and orientation of two three- 
dimensional objects therein consisting of the further steps of: f) determining a third 
landmark on either said three-dimensional object or on said other object; and g) 
further determining the pose of said three-dimensional object with respect to said 
first, second and third landmarks. 

In a yet further aspect of the invention a method of analysis of a two- 
dimensional image to determine the position and orientation of two three- 
dimensional objects therein consisting of the further steps of: h) repeating steps a to g 
at another time; and i) comparing poses obtained in step g) to determine the relative 
movement of said three-dimensional object. 

In a broad aspect of the invention an apparatus for the analysis of a two-dimensional 
image to determine the position and orientation of a three-dimensional object therein 
consists of; a shape determining means for creating a silhouette of said three- 
dimensional object in said two-dimensional image; and a three-dimensional pose 
estimation means which matches a three-dimensional representation of said object to 
the two-dimensional silhouette of said object in said two-dimensional image. 

In a further broad aspect of the invention an apparatus for analysis of a two- 
dimensional image comprises; a two-dimensional prosthesis pose estimation means 
for identifying a predetermined prosthesis in said two-dimensional image; a two- 
dimensional bone pose estimation means for identifying landmarks on said bone; 
and a two-dimensional measure determination means for determining a scale for 
said two-dimensional image and a measure of the relative distance of one or more of 
said landmarks from predetermined points on said prosthesis. 

In a further aspect of the invention said apparatus for analysis further 
comprises; a three-dimensional prosthesis pose estimation means for identifying a 
predetermined prosthesis in said three-dimensional image; a three-dimensional bone 
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pose estimation means for identifying a bone pose; and a three-dimensional 
measures determination means for determining the relative pose of said bone to said 
prosthesis. 

In a yet further aspect of the invention wherein over time the three- 
dimensional measures determination means provides the components of axial 
rotation, rotational and/ or distal migration of the complex movement of said 
prosthesis relative to said bone. 

Specific embodiments of the invention will now be described in some further 
detail with reference to and as illustrated in the accompanying figures. These 
embodiments are illustrative, and not meant to be restrictive of the scope of the 
invention. Suggestions and descriptions of other embodiments may be included 
within the scope of the invention but they may not be illustrated in the 
accompanying figures or alternatively features of the invention may be shown in the 
figures but not described in the specification. 

BRIEF DESCRIPTION OF THE FIGURES 

Fig. 1 depicts a functional block diagram of an apparatus that is in accord with 
a method of analysis of a two-dimensional image to determine the relative two- 
dimensional position between a bone and an implanted prosthesis within the image; 

Fig. 2 depicts a functional block diagram of the apparatus and method of Fig 1 
supplemented with an apparatus that automates the analysis of a two-dimensional 
image to determine the relative two-dimensional position between a bone and an 
implanted prosthesis; 

Fig. 3 depicts a functional block diagram of the apparatus of Figs 1 and 2 
supplemented so that relative distances between the bone and prosthesis can be 
determined and outputs data that provides the basis for a three-dimensional 
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presentation of the relationship between the position of a bone and an implanted 
prosthesis; 

Fig. 4 depicts a graph showing a migration history of the anterior posterior 
distal movement of a prosthesis relative to a bone; 

Fig. 5 depicts a representation of a methodology for improving the relative 
measurement accuracy of the apparatus. 

Fig. 6 depicts the nature of the possible migration and rotation of a prosthesis; 

Fig. 7 depicts a bone and prosthesis example and in particular identifies 
specific features of the said prosthesis; and 

Fig. 8 depicts examples of prosthesis shoulder position determination 
comparing prior methods and the method of the invention. 

DETAILED DESCRIPTION OF AN EMBODIMENT OF THE INVENTION 

It should be understood that although the embodiment conveniently 
compartmentalizes the apparatus and proscribes a complementary method, it does 
not limit the ability for the invention to be provided by a different arrangement of the 
same or different functions using alternative methods. 

Neither is the invention limited in its application to the analysis of the 
bonding of medically implanted orthopaedic prosthesis to human bone as is 
disclosed in detail herein. 

The inventors have identified and describe by way of example, an apparatus 
and method for the determination of the relative orientation of bone and implant, 
and over time track the effectiveness of bonding between the two. In this 
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specification and in particular in the description of the embodiment, the two- 
dimensional and three-dimensional position and orientation of the bone and implant 
will be referred to as its "pose". 

In one aspect of the invention as disclosed in this embodiment, there has been 
identified a need to determine the three-dimensional position and orientation of an 
implant with respect to the surrounding bone using a single 2-dimensional view of 
those objects. 

A patient having bone or joint problems presents to a clinician, X-rays and 
three-dimensional representations of the relevant body portion (including the 
affected bone) are obtained almost as a matter of course as an aid to the diagnosis of 
the problem. 

An analysis of the medical condition of the patient may conclude the need for 
surgery and their suitability for prosthesis implant is undertaken and a suitable 
prosthesis is chosen. As part of the preparation for use of this embodiment of the 
invention, a three-dimensional digital representation of the chosen prosthesis may 
also be created but this is not always done. 

The prosthesis placement operation is performed and preferably during the 
following months X-rays are taken at convenient times. The taking of a X-ray 
immediately following the operation greatly improves the detection of early failure 
of the bonding between bone and implant but X-rays become less frequent if no 
problems to the patient present themselves. 

It is those 2-d X-rays taken over time and the three-dimensional prosthesis and 
bone data that can be used by the present invention to provide automatic and semi- 
automatic analysis of the bone /prosthesis union and other characteristics. The 
consequent presentation by the apparatus of the invention of prosthesis migration 
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information is useful for the clinician in assessing the condition over time of the 
union between bone and the prosthesis. 

The inventors have determined that at least in this embodiment, it is 
preferable to break down the analysis process into steps. 

As stated previously, an X-ray image of the bone and implant is cheap relative 
to other types of medical imaging. Taking an X-ray of the patient following surgery 
will ideally provide a well-contrasted shape of the outline of a solid three- 
dimensional object, such as an implant against the bone it has been surgically placed 
on or into. This X-ray can form the basis for a particular type of analysis by the 
apparatus of the invention that will firstly identify the outline (silhouette) of the 
implant. 

The identification of the bone in the X-ray is also a necessary task but will be 
described later in the specification. 

The X-ray machine may produce a film which needs to be digitized or it may 
produce a digital image which itself can be used to produce a film image. Having an 
X-ray image provides a view of the prosthesis from only one perspective. One of the 
main tasks identified by the inventors in providing the invention is the determination 
of the actual outline of the solid object (typically the prosthesis) solely from the 
typically gray scale image information contained in a digitized X-ray image. 

There exist a number of processes for isolating /identifying a feature boundary 
in a two-dimensional image. 

In one embodiment, any line shape (circle, ellipses, lines) in a two-dimensional 
image can be identified using a Hough transform. This is merely a preferred method 
for achieving this aim, as there are a number of other processes that could also be 
used. In the example of hip joint replacement prosthesis, shapes such as the ball and 
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edges of the stem of the implant can be identified most easily. The remainder of the 
implant can then be identified using other methods and known geometric 
characteristics of the implant itself. 

Having determined the outline of the implant, the next step is to determine the 
three-dimensional spatial orientation of the implant that would give that outline. 

There also exist a number of different processes that can provide this 
determination but in this embodiment, the approach is to use a modified Expectation 
Maximization algorithm and an alternative method is the use of the Hausdorff 
distance. An Expectation Maximization (EM) algorithm and the Hausdorff distance 
method are iterative processes for estimating truth from limited data. 

The EM algorithm is statistical in nature and deals with probability density 
functions for unknown variables, whereas the Hausdorff distance is geometrical in 
nature and compares the measured distances between a plurality of predetermined 
points on both the image and the actual three-dimensional shape of the implant 

* 

which has been previously digitized. 

In this embodiment, an initial estimate of the pose of the implant is made. 
Once made, the apparatus assesses if a change in the pose will "improve" the fit of 
the silhouette of a three-dimensional implant that is evident in the digital X-ray 
image. The apparatus then selects a change and assesses whether the new pose 
created by the change improves the fit. The method then iterates these steps until 
convergence occurs. 

The actual implementation and modification to an Expectation Maximization 
algorithm may include modifications to both the Generalized Expectation 
Maximization algorithm and the Expectation-Conditional Maximization algorithm 
such as the inclusion of nuisance variables (which have to be integrated out) related 
to the prosthesis modeling mesh as well as constraining pose changes to real 
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rotations in three-dimensional space. These modifications are merely preferable and 
it is with in the skill of one in the art to implement both the EM algorithm and any 
modifications that are deemed useful. 

A yet further method for determining the pose of the three-dimensional object 
(prosthesis) in a digitized two-dimensional image is to use an area-matching 
algorithm. The area occupied by the three-dimensional object in the two-dimensional 
image can be equated to, for example, the number of picture element used to store 
and display it, while the number of picture elements used to display the 
representation of the actual three-dimensional object at different poses, can also be 
determined. The area-matching algorithm can be implemented in a number of ways 
and preferably a clipped region minimization process is used to align the silhouette 
of the three-dimensional object in the two-dimensional image with pose of the 
representations of the prosthesis. Again this process is implement in steps that iterate 
until convergence occurs. 

Once the pose of the implant has been determined from the two-dimensional 
X-ray image, it is then possible to use the same processes to determine the pose of the 
bone and thus determine the relative orientation in three dimensions of the implant 
to the bone. This is one way to establish a base line for future comparisons of the 
bone to implant union. 

Alternatively, (either with the assistance of a clinician or by automatic image 
analysis techniques) a reference point (landmark /s) can be identified on the bone and 
determination can then be made in two dimensions of the position of the implant 
relative to the landmark. Likewise this is another way to establish a base line. 

Most importantly, it does not matter whether the reference points are located 
on or simply related to the respective objects. That is, it is possible for reference 
points to be any point in the co-ordinate system of either object or an actual point on 
the object, which of course, in any event, is a point in a co-ordinate system of choice. 
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Distance measurements can be made between co-ordinate systems using one 
of them as a reference or a common origin of the two co-ordinate systems can be 
created. There are many ways in which the relative movement of the objects can be 
measured, that is, their orientation as well for the future their relative displacement 
over time. 

Absolute measurements (with some error) can be determined by knowing the 
actual dimensions of the prosthesis and the use of a scaling factor. 

The invention provides much promise that clinicians will be encouraged to 
take relatively cheap X-rays of their patient's implants at appropriate intervals 
following surgery, to benefit from the increased bone /prosthesis bonding analysis 
that is possible using the invention. The two-dimensional information from an X-ray 
will form the basis for an automatically generated bone to implant bonding 
assessment which will include the ability to flag the need for more detailed analysis 
of potential problem areas. The apparatus and method of the invention can provide 
automatic, repeatable and consistent pose estimation of the bone and implant as well 
as a measure of the relative movement (lateral as well as rotational) between them. 

Fig 1 depicts the first three main parts of the apparatus that is common to all 
other parts. 

A means 10 is used to input a two-dimensional digital image to the apparatus. 
The image of the bone and implant of a patient is recorded. One way in which this is 
done, is to X-ray the relevant part of the patient. The X-ray image is digitized at a 
predetermined resolution direct so as to preserve as many details as possible of the 
image. The preferred manner is to generate at the X-ray machine a digital X-ray 
image of the patent having the highest manageable resolution. Some post digitizing 
filtering can be performed to clean up the image to better define the outlines of solid 
objects most particularly the prosthesis. Elimination of extraneous printed 
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information on the image and any standard imperfections created by the imaging 
process are just some of the filtering that can be done. Such filters are well known in 
the art. 

The second of the three elements of the apparatus shown in Fig. 1 is a means 
12 to retrieve from digital storage, either or both of a three-dimensional digital model 
of the bone of the same patient that was digitized prior to the surgery and/or a three- 
dimensional model of the prosthesis, the later of the two being the most useful. 

The three-dimensional image of the patient's bone or even a suitably 
manipulated contra-limb is preferably obtained from a CAT scan but it may also be 
possible to use a standardised bone model. The later would obviate taking the CAT 
scan. The raw scan is used to create a surface model in a digital-modeling format that 
is consistent within the remainder of the apparatus and method, and such techniques 
are within the skill of those in the art. 

The three-dimensional model of the implant is created from the implant 
manufacturing files or recreated from the actual object and is typically in the form of 
a "net" of control points located on a virtual surface. The file representing those 
control points is provided in a digital modeling format that will be consistent within 
the remainder of the apparatus and method. 

The two data files are stored in the historical data block 12 and are available 
for acceptance and manipulation by the two-dimensional pose estimator 14 which is 
the third of the elements of the first part of the apparatus disclosed in Fig. 1. 

The two-dimensional pose estimator 14 may use a variety of techniques for 
determining pose, some of which have been described previously. Preferably edge 
detection algorithms are used to identify boundaries in the image having particular 
characteristics. For example, a Hough transform can be used and the simplest shapes 
to identify in most prosthesis are their straight lines. The next simplest shape to be 
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identified depends on what type of prosthesis it is. As an example only, a hip joint 
replacement prosthesis has a spherical shape at one end that takes the place of the 
head of the femur in the hip joint. Thus, a sphere or circular shape will be readily 
identifiable. Generally, prosthesis shape identification is simple because the 
prosthesis has regular and consistent shape. 

Where appropriate it is also possible to use a subtractive clipped region 
process as the primary or even as a secondary confirmatory process or even in 
combination with other processes to align the prosthesis silhouette so as to determine 
its pose. 

As will be described later, it is also possible to advantageously improve the 
speed of determining a three-dimensional pose of an object by using as a starting 
point, the results of the two-dimensional pose 14 estimation process 22, as is depicted 
in Fig. 3 by the dotted line 15. However, a random starting point would also suffice 
however convergence will likely not be as quick as a prior process based estimation. 

Furthermore, it is possible to further improve the two-dimensional 
measurement process 16 for determining relative position (distance) and orientation 
by using the results of the three-dimensional pose estimation of the prosthesis 22, as 
is depicted by the connection created by the dashed line 23 in Fig. 3. 

Fig 1 depicts a two-dimensional measure means 16 that receives the spatial 
data available from the two-dimensional pose estimation means and calculates a 
scaling factor with the available information. The means 16 knows the actual 
dimensions of the prosthesis and can compare it with the two-dimensional image 
dimensions and thereby provide a scale for absolute and relative measurements on 
the two-dimensional image. Such measurements can be used when combining the 
method and apparatus of this invention with known EBRA methods and 
independently generated guidelines for comparability of two-dimensional images. 
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Fig 2 depicts a manual two-dimensional pose estimation means 20 which 
allows a clinician or other skilled person to identify landmarks on the two- 
dimensional representation of the bone surrounding the implanted prosthesis (in this 
example, a femur). Preferably, more than one and likely 10-20 landmarks are 
identified. The use of easily identifiable landmarks is important, otherwise, as is the 
problem with prior arrangements, there will be a lack of comparability of images 
over time. As will be described later, it is also possible to use the three-dimensional 
representation of the bone before surgery to assists in the determination of relative 
position and orientation of the bone and prosthesis. No one landmark should be 
relied upon as clinical variance can be the cause of inconsistent results. Not 
unsurprisingly this situation can arise from a number of circumstances, for example, 
the x-rays separated in time will have slightly different perspectives of the same bone 
and land marks may become obscured and misidentification can occur; bone will 
grow and some land marks may move as a result; the accuracy associated with 
digital image manipulation will impinge upon a humans ability to readily identify a 
specific point in the 2-dimensional plane which is consistent over time. 

Once an adequate number of landmarks have been identified, the form of the 
bone surrounding the prosthesis can be defined by that set of landmarks. Thus in the 
two-dimensional measure means 16 (Figs 1 and 2), the relative distance and known 
landmarks on the prosthesis from one or more of the landmarks can be calculated. If 
more than one X-ray image is available and the manual pose estimation step 
performed in 20 can be done for each, it is possible to more accurately determine the 
relative distances involved. This is in preparation for the time that the current 
determination is to be compared with historical image data determinations. The 
distances between landmarks on the femur and the prosthesis that are likely to have 
changed can be compared recognising that all measurement systems have error 
characteristics unique to that system. 

Clearly, changes in relative distances are an indication of the migration of the 
prosthesis and these differences, if they exist or not, are illustrated using the 2-d 
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diagnosis presentation process in 18. Typically, quantification of migration overtime 
is of greatest interest to clinicians and a graphical presentation not unlike that of Fig 4 
is most useful. It is of course a clinical decision as to whether the quantification in 
light of any margins for accuracy error are indicative of acceptable or unacceptable 
migration for that particular patient. 

It is however, advantageous to automate these processes using historical data 
12 of the bone prior to or following surgery to provide the bases for the two- 
dimensional pose estimation of the bone. Fitting the three-dimensional bone form 
information into the two-dimensional image containing some portion of the bone is 
the first step. Account can be taken for changes to the bone that results from the 
surgery, such as the absence of the bone or additional growths following surgery. 

Fig 3 depicts the third part of the apparatus. 

Again using historical data the three-dimensional form of the prosthesis can 
facilitate advantageous use of the estimation methodologies described above in 
matching the projected silhouette of the three-dimensional prosthesis model with the 
silhouette of the extracted two-dimensional image of the prosthesis. The process 
conducted in 22 is a three-dimensional pose estimation of the prosthesis. 

Again, it is also necessary to initially estimate the three-dimensional pose of 
the bone (femur in this example), using the same techniques discussed in relation to 
the two-dimensional pose estimation conducted in means 14 as alluded to earlier and 
indicated by dotted line 15. As an example only, the two-dimensional pose 
estimation can be used as a starting point for the three-dimensional pose estimation. 
The two-dimensional pose estimation is a likely orientation of the prosthesis, which 
is unlikely to be any more than 90° out along any axis. Therefore the use of the two- 
dimensional pose estimation as a starting point for the three-dimensional pose 
estimation can save a great deal of calculation. Choice of a 3-dimensional co-ordinate 
system for this process will likely have coincidence with the 2-dimensional co- 
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ordinate system of the 2-d pose estimation and 2-d measures processes but need not 
necessarily be so. When comparisons are required suitable translations can be 
provided. 

Not unlike the steps taken with element 20, historical data can be used along 
with the pose estimation methodologies described above to create a 3-d pose 
estimation of the bone in 24. Once the three-dimensional pose estimation of the bone 
(in this embodiment the femur) is complete, it is possible to calculate distances 
between one or more landmarks or the two objects, one being the bone and the other 
being the prosthesis using the 3-d measures processes in 26 for presentations using 
the 3-diagnosis process 28. 

Therefore, it will be possible to not only accurately determine the orientation 
of one element (the prosthesis) relative to another element (bone /femur), it is 
possible to calculate migration in all the axes of possible motion and thus much 
better represent for the clinician the relative migration of the elements over time. 

As depicted in Fig. 6 there are two types of migration (lateral with respect to 
the longitudinal axis of the implant and distal typically along the longitudinal axis of 
the implant) and one type of rotation (axial). It is the manipulation of the above 
calculation results into graphs or animations when presented to clinicians that will 
provide the greatest assistance to them. They will be better able to assess the bone 
prosthesis union characteristics and the result can then be relied upon for 
consideration as to whether an expensive CAT scan should be done and then based 
on all the information at hand whether further surgery should be contemplated. 

Fig 4 is one example of a graph that discloses the migration of an implanted 
prosthesis over time in the distal direction of an AP view. The vertical axis displays 
migration in millimetres while the horizontal axis displays years following the 
operation involving the implantation of the prosthesis into the bone. 
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The intervals between X-rays are not regular as is indicated by the crosses that 
denote the value of migration in millimetres at particular times following the 
operation. As would be expected the majority of movement occurs soon after the 
operation, while the degree of movement falls within a relatively narrow distance 
boundary as time progresses. Marked fluctuations outside an acceptable clinical 
variation will likely confirm patient experience and alert the clinician, sometimes 
though even without patient indications, that the bone prosthesis union is 
deteriorating. Fig 4 is but one example of a way to depict the results of 2-d and 3-d 
measurements taken from images collected overtime. Error bars could be added and 
there could be use of representative images and measurements as well as expert 
systems information in the form of clinically acceptable migration boundaries 
overtime. Features of relevance to a clinician can be identified and highlighted by 
using look up tables and rules. 

Fig 5 depicts a proposed approach for improving the two-dimensional and 
three-dimensional pose estimation processes by taking more than one X-ray record of 
the same limb containing the prosthesis from different angles about the prosthesis. 
Using the two-dimensional and three-dimensional pose estimation techniques 
described above for both the prosthesis and the surrounding bone allows multiple 
calculations to occur and subsequently the two-dimensional measures as well as the 
three-dimensional relative distance and orientation measures can help to improve in 
accuracy. 

There exist a number of techniques for analyzing these results but the simplest 
is to use an arithmetic mean of results to provide what will become clinically 
acceptable measures, distances and orientations. 

Prior art two-dimensional techniques determine the migration of a prosthesis 
implant using the change in position, of a femoral landmark, along the axis of the 
stem of the prosthesis, relative to a fixed point on the stem. The point regularly 
chosen is the shoulder of the stem - in other words, the edge of the stem, as viewed 
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in the X-ray, where its axis crosses the stem edge (refer Fig. 7). The tip and shoulder 
are defined with respect to the intersection points of the outer edge of the stem of the 
prosthesis silhouette and lie along the bisector of the prosthesis stem. The shoulder is 
a useful reference point on the prosthesis for use in migration determinations, as long 
as in prior art determinations the view of the prosthesis is taken from a consistent 
angle about the stem's axis. 

However, as the view of the stem rotates about its axis (i.e. X-rays taken from 
a more lateral than AP view) the shoulder point that a two-dimensional 
determination alone provides, actually moves up towards the ball. In particular, 
when the X-ray is taken from a lateral view, the shoulder cannot be seen at all, as the 
ball of the implant obscures it. In this case, the images cannot be used for tracking. 
The difference between a prior art determination of the location of a shoulder is 
compared with the actual position of the shoulder and is depicted in Fig. 8. 

The method and apparatus of the invention determines the rotation of the 
view of the prosthesis about its axis, which, amongst other determinations, enables 
determination of an exact three-dimensional view of the prosthesis. By mapping the 
prosthesis model back on to the image, it is thus possible to determine the exact 
position of the axis and shoulder point of the prosthesis in the image. Therefore it is 
possible to more accurately determine the actual migration of the shoulder with 
respect to another landmark on the bone of the patient. Also, with the determinations 
made using the method and apparatus of the invention, all X-rays, whether AP or 
lateral, can be used to calculate any amount migration that may have occurred. 

As is described as a preferable approach, a single, manually marked reference 
point on the femur can be used to calculate migration relative to the shoulder point 
on the prosthesis. The point is either derived from the two-dimensional techniques 
of the invention or makes use of the three-dimensional pose determination to give a 
better 2-dimensional result. The error due to manual point marking is approximately 
in the range of ±0.4mm. This has been calculated using the fact that the marker can be 
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placed 2 to 3 picture elements away from the desired landmark, and that the images 
are generally processed at a resolution of around 0.1 to 0.2mm per picture elements. 

It will be appreciated by those skilled in the art, that invention is not restricted 
in its use to the particular application described. Neither is the present invention 
restricted as preferred embodiment with regard to the particular elements and /or 
features described or depicted herein. It will be appreciated that various 
modifications can be made without departing from the principles of the invention. 
Therefore, the invention should be understood to include all such modifications 
within its scope. 
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THE CLAIMS DEFINING THE INVENTION ARE AS FOLLOWS: 

1. A method for analysis of a two-dimensional image to determine the position 
of a three-dimensional object therein consisting of the steps; 

a) determining the form of said three-dimensional object in the two- 
dimensional image; 

b) determining the pose of a three-dimensional representation of said 
object that conforms to the form of the said three-dimensional object in 
the two-dimensional image; 

c) determining a first landmark with reference to said three-dimensional 
object; 

d) determining a second landmark on an other object in said two- 
dimensional image; and 

e) determining the distance between said landmarks. 

2. A method according to claim 1 consisting of the further steps of; 

f) determining a third landmark on either said three-dimensional object 
or on said other object; and 

g) further determining the pose of said three-dimensional object with 
respect to said first, second and third landmarks. 

3. A method according to claim 2 consisting of the further steps of; 

h) repeating steps a to g at another time; and 

i) comparing poses obtained in step g) to determine the relative 
movement of said three-dimensional object. 

4. A method according to claim 3 where in step a) the form is determined using 
an edge detection algorithm to identify the silhouette of predetermined shapes of 
said three-dimensional object in the two-dimensional image. 
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5. A method according to claim 3 where in step b) the pose is determined by 
iterative matching of the area of the two-dimensional image of the three-dimensional 
object in the two-dimensional image against the area of the three-dimensional 
representation of the three-dimensional object in different poses. 

6. A method of analysis of a two-dimensional image to determine the position 
and orientation of two three-dimensional objects therein consisting of the steps of: 

a) determining the form of both three-dimensional objects in the two 
dimensional image; 

b) determining the pose of both three-dimensional objects; 

c) determining the relative pose of one said object to the other. 

7. A method according to claim 6 consisting of the further steps of; 

d) repeat steps a to c at another time; and 

e) comparing the relative pose obtained in step d) with the relative pose 
obtained in step c) to determine the relative movement of said three- 
dimensional objects. 

8. An apparatus for the analysis of a two-dimensional image to determine the 
position and orientation of a three-dimensional object therein consists of; 

a shape determining means for creating a silhouette of said three-dimensional 
object in said two-dimensional image; and 

a three-dimensional pose estimation means which matches a 
three-dimensional representation of said object to the two-dimensional silhouette of 
said object in said two-dimensional image. 

9. An apparatus for analysis of a two-dimensional image according to claim 8 
further consists of; 

a two-dimensional prosthesis pose estimation means for identifying a 
predetermined prosthesis in said two-dimensional image; 
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a two-dimensional bone pose estimation means for identifying landmarks on 
said bone; and 

a two-dimensional measure determination means for determining a scale for 
said two-dimensional image and a measure of the relative distance of one or more of 
said landmarks from predetermined points on said prosthesis. 

10. An apparatus for analysis of a two-dimensional image according to claim 9 
further consists of: 

a three-dimensional prosthesis pose estimation means for identifying a 
predetermined prosthesis in said three-dimensional image; 

a three-dimensional bone pose estimation means for identifying a bone pose; 

and 

a three-dimensional measures determination means for determining the 
relative pose of said bone to said prosthesis. 

11. An apparatus for analysis of a two-dimensional image according to claim 10 
wherein over time the three-dimensional measures determination means provides 
the components of axial rotation, rotational and /or distal migration of the complex 
movement of said prosthesis relative to said bone. 
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